An Average-case Analysis of the Gaussian Algorithm for Lattice Reduction an Average-case Analysis of the Gaussian Algorithm for Lattice Reduction an Average-case Analysis of the Gaussian Algorithm for Lattice Reduction

The Gaussian algorithm for lattice reduction in dimension 2 is analysed under its standard version. It is found that, when applied to random inputs in a continuous model, the complexity is constant on average, the probability distribution decays geometrically, and the dynamics is characterized by a conditional invariant measure. The proofs make use of connections between lattice reduction, continued fractions, continuants, and functional operators. Analysis in the discrete model and detailed numerical data are also presented. Une analyse en moyenne de l'algorithme de Gauss de r eduction des r eseaux R esum e. L'algorithme de r eduction des r eseaux en dimension 2 qui est d^ u a Gauss est analys e sous sa forme dite standard. Il est etabli ici que, sous un mod ele continu, sa complexit e est constante en moyenne et que la distribution de probabilit es associ ee decro^ t g eom etriquement tandis que la dynamique est caract eris ee par une densit e conditionnelle invariante. Les preuves font appel aux relations entre r eduction des r eseaux, fractions continues, continuants, et op erateurs fonctionnels. Une analyse du mod ele discret compl et ee de donn ees num eriques est aussi pr esent ee. Abstract. The Gaussian algorithm for lattice reduction in dimension 2 is analysed under its standard version. It is found that, when applied to random inputs in a continuous model, the complexity is constant on average, the probability distribution decays geometrically, and the dynamics is characterized by a conditional invariant measure. The proofs make use of connections between lattice reduction, continued fractions, continuants, and functional operators. Analysis in the discrete model and detailed numerical data are also presented.